Blind Separation of Dependent Sources and Subspaces by Minimum Mutual Information

We consider the problem linear separation of dependent sources by minimization of mutual information. We define a type of generalized variance dependence for random vectors, which can be analyzed into what we call “level curve dependence” and “envelope dependence”. We define subgaussian dependence and supergaussian dependence in terms of convexity with respect to the quadratic, and define the “homogeneous dependence” types, sub-sub and sup-sup, and the “conflicting” types sub-sup and sup-sub, in terms of the respective convexity of the envelope and level curve functions. We show that mixtures of subspaces of homogeneously dependent sources can be separated by minimizing the total output mutual information, without requiring a priori knowledge of the subspace structure, and we show that conflicting dependence types are actually separated by maximizing mutual information, creating a problem for subspace separation by simple minimization of output mutual information. Monte Carlo simulations are provided verifying the theory.